In order to solve the problems of control input chattering of traditional sliding mode, requiring acceleration term, and limited application model of traditional disturbance observes in manipulator joint position tracking, a self-adaptive inverse sliding mode control algorithm for manipulators with improved nonlinear disturbance observer was proposed. Firstly, an improved nonlinear disturbance observer to perform on-line testing was designed. In the sliding mode control law, interference estimates were added to compensate for observable disturbance, and then appropriate design parameters were selected to make the observation error converge exponentially; the adaptive control law was performed to estimate the unobservable interference and further improved the tracking performance of the control system. Finally, the Lyapunov function was used to verify the asymptotic stability of the closed-loop system, then the system was applied to the joint position tracking of the manipulator. The experimental results show that compared with the traditional sliding mode algorithm, the improved control algorithm not only accelerates the response speed of the system, but also effectively suppresses the system chattering, avoids measuring acceleration items and expands the application scope of the application model.